MSDA - Bootcamp 2025 Summer
KT Wong
kwanto@hku.hk
Faculty of Social Sciences, HKU
Aug 25, 2025
The materials in this topic are drawn from Stachurski (2016)
Vectors
Matrix operations
Linear Transformation
System of Linear Equations
Vector Spaces
Eigenvalues and Eigenvectors
Applications for calculus
\[ u=\begin{bmatrix} 1 & 0 \end{bmatrix} \]
\[ v=\begin{bmatrix} 1 \\ 0 \end{bmatrix} \]
The dot product of vectors \(x,y \in \mathbb R^n\) is defined as
\[ \begin{array}{rl} x^\top y &= {\color{red}{x_1 y_1}} + {\color{blue}{x_2 y_2}} + \cdots + x_n y_n \\ &= \sum_{i=1}^n x_i y_i \end{array} \]
the dot product could be written as \(x \cdot y\) or \(x'y\)
the dot product could be expressed in cosine form
\[ x \cdot y = \|x\| \|y\| \cos \theta \]
\[ A + B = \begin{bmatrix} a_{11} & \cdots & a_{1k} \\ \vdots & \vdots & \vdots \\ a_{n1} & \cdots & a_{nk} \end{bmatrix} + \begin{bmatrix} b_{11} & \cdots & b_{1k} \\ \vdots & \vdots & \vdots \\ b_{n1} & \cdots & b_{nk} \end{bmatrix} = \begin{bmatrix} a_{11} + b_{11} & \cdots & a_{1k} + b_{1k} \\ \vdots & \vdots & \vdots \\ a_{n1} + b_{n1} & \cdots & a_{nk} + b_{nk} \end{bmatrix} \]
scale multiplication
\[ \gamma A = \gamma \begin{bmatrix} a_{11} & \cdots & a_{1k} \\ \vdots & \vdots & \vdots \\ a_{n1} & \cdots & a_{nk} \end{bmatrix} := \begin{bmatrix} \gamma a_{11} & \cdots & \gamma a_{1k} \\ \vdots & \vdots & \vdots \\ \gamma a_{n1} & \cdots & \gamma a_{nk} \end{bmatrix} \]
The rule for matrix multiplication generalizes the idea of inner products
Here’s an example of a \(2 \times 2\) matrix multiplied by a \(2 \times 1\) vector
\[ Ax = \begin{bmatrix} \color{red}{a_{11}} & \color{red}{a_{12}} \\ a_{21} & a_{22} \end{bmatrix} \begin{bmatrix} \color{red}{x_1} \\ \color{red}{x_2} \end{bmatrix} = \begin{bmatrix} \color{red}{a_{11}x_1 + a_{12}x_2} \\ a_{21}x_1 + a_{22}x_2 \end{bmatrix} \]
\[ AB = \begin{bmatrix} a_{11} & a_{12} \\ \color{red}{a_{21}} & \color{red}{a_{22}} \\ \end{bmatrix} \begin{bmatrix} b_{11} & \color{red}{b_{12}} \\ b_{21} & \color{red}{b_{22}} \\ \end{bmatrix} := \begin{bmatrix} a_{11}b_{11} + a_{12}b_{21} & a_{11}b_{12} + a_{12}b_{22} \\ a_{21}b_{11} + a_{22}b_{21} & \color{red}{a_{21}b_{12} + a_{22}b_{22}} \end{bmatrix} \]
Notice that \(AB \neq BA\) in general
There are many tutorials to help you further visualize this operation, like this one
One important special case is the identity matrix, which has ones on the principal diagonal and zero elsewhere:
\[ I = \begin{bmatrix} 1 & \cdots & 0 \\ \vdots & \ddots & \vdots \\ 0 & \cdots & 1 \end{bmatrix} \]
\[ Y=\begin{bmatrix} 3 & 1 & -2 \\ 6 & 3 & 4 \end{bmatrix} , \quad X=\begin{bmatrix} 4 & 2 \\ 3 & 0 \\ 1 & 2 \end{bmatrix} \]
What are the dimensions of the matrices \(X\) and \(Y\)?
How to multiply \(Y\) and \(X\) to yield a \(3 \times 3\) matrix? Compute it
How to multiply \(Y\) and \(X\) to yield a \(2 \times 2\) matrix? Compute it
\[ f(\alpha x + \beta y) = \alpha f(x) + \beta f(y) \]
We consider how a given matrix transforms in \(\mathbb R ^2\)
Let \(\begin{bmatrix} 2 & 1 \\ -1 & 1 \end{bmatrix}\)
Consider the two matrices
\[ A = \begin{bmatrix} 0 & 1 \\ -1 & 0 \end{bmatrix} \quad \text{and} \quad B = \begin{bmatrix} 1 & 2 \\ 0 & 1 \end{bmatrix} \]
What will the output be when we try to obtain \(ABx\) for some \(2 \times 1\) vector \(x\)?
\[ \color{red}{\underbrace{ \color{black}{\begin{bmatrix} 0 & 1 \\ -1 & 0 \end{bmatrix}} }_{\textstyle A} } \color{red}{\underbrace{ \color{black}{\begin{bmatrix} 1 & 2 \\ 0 & 1 \end{bmatrix}} }_{\textstyle B}} \color{red}{\overbrace{ \color{black}{\begin{bmatrix} 1 \\ 3 \end{bmatrix}} }^{\textstyle x}} \rightarrow \color{red}{\underbrace{ \color{black}{\begin{bmatrix} 0 & 1 \\ -1 & -2 \end{bmatrix}} }_{\textstyle AB}} \color{red}{\overbrace{ \color{black}{\begin{bmatrix} 1 \\ 3 \end{bmatrix}} }^{\textstyle x}} \rightarrow \color{red}{\overbrace{ \color{black}{\begin{bmatrix} 3 \\ -7 \end{bmatrix}} }^{\textstyle y}} \]
\[ \color{red}{\underbrace{ \color{black}{\begin{bmatrix} 0 & 1 \\ -1 & 0 \end{bmatrix}} }_{\textstyle A} } \color{red}{\underbrace{ \color{black}{\begin{bmatrix} 1 & 2 \\ 0 & 1 \end{bmatrix}} }_{\textstyle B}} \color{red}{\overbrace{ \color{black}{\begin{bmatrix} 1 \\ 3 \end{bmatrix}} }^{\textstyle x}} \rightarrow \color{red}{\underbrace{ \color{black}{\begin{bmatrix} 0 & 1 \\ -1 & 0 \end{bmatrix}} }_{\textstyle A}} \color{red}{\overbrace{ \color{black}{\begin{bmatrix} 7 \\ 3 \end{bmatrix}} }^{\textstyle Bx}} \rightarrow \color{red}{\overbrace{ \color{black}{\begin{bmatrix} 3 \\ -7 \end{bmatrix}} }^{\textstyle y}} \]
We can observe that applying the transformation \(AB\) on the vector \(x\) is the same as first applying \(B\) on \(x\) and then applying \(A\) on the vector \(Bx\).
the matrix product \(AB\) is the composition of the matrix transformations \(A\) and \(B\)
Let \(A\) be the \(90^{\circ}\) clockwise rotation matrix given by \(\begin{bmatrix} 0 & 1 \\ -1 & 0 \end{bmatrix}\)
Let \(B\) be a shear matrix along the x-axis given by \(\begin{bmatrix} 1 & 2 \\ 0 & 1 \end{bmatrix}\)
visualize how a grid of points changes when we apply the transformation \(AB\) and then compare it with the transformation \(BA\)
In economics, we are often interested in analyzing behavior where we repeatedly apply a fixed matrix
For example, given a vector \(v\) and a matrix \(A\), we are interested in studying the sequence
\[ v, \quad Av, \quad AAv = A^2v, \quad \ldots \]
Example of a sequence of iterates \((A^k v)_{k \geq 0}\) under map \(A=\begin{bmatrix} \sqrt{3}+1 & -2 \\ 1 & \sqrt{3}-1 \end{bmatrix}\)
let \(v = (-3, -3)\)
Example of a sequence of iterates \((A^k v)_{k \geq 0}\) under map \(A=\begin{bmatrix} \sqrt{3}+1 & -2 \\ 1 & \sqrt{3}-1 \end{bmatrix}\)
let \(v = (2.5, 0)\)
Example of a sequence of iterates \((A^k v)_{k \geq 0}\) under map \(A=\begin{bmatrix} \sqrt{3}+1 & -2 \\ 1 & \sqrt{3}-1 \end{bmatrix}\)
let \(v = (-1, -0.25)\)
\[ \begin{aligned} y_1 = a x_1 + b x_2 \\ y_2 = c x_1 + d x_2 \end{aligned} \]
A more general version looks as follows
\[ \begin{matrix} a_{11} x_1 & + & a_{12} x_2 & + & \cdots & + & a_{1n} x_n & = & b_1 \\ \vdots & & \vdots & & & & \vdots & & \vdots \\ a_{n1} x_1 & + & a_{n2} x_2 & + & \cdots & + & a_{nn} x_n & = & b_n \end{matrix} \]
The objective here is to solve for the “unknowns” \(x_1, \ldots, x_n\)
We take as given the coefficients \(a_{11}, \ldots, a_{nn}\) and constants \(b_1, \ldots, b_n\)
this is a setting where the number of unknowns equals the number of equations
\[ A x = b \quad \text{where} \quad A = \begin{bmatrix} a_{11} & \cdots & a_{1n} \\ \vdots & \vdots & \vdots \\ a_{n1} & \cdots & a_{nn} \end{bmatrix} \quad \text{and} \quad b = \begin{bmatrix} b_1 \\ \vdots \\ b_n \end{bmatrix} \] - find a vector \(x \in \mathbb R^n\) that solves the equations, taking \(b\) and \(A\) as given
Consider the system of equations given by
Consider the system of equations given by
\[ \begin{aligned} x - 2y &= -4 \\ -2x + 4y &= 8 \end{aligned} \]
Any vector \(v = (x,y)\) such that \(x = 2y - 4\) will solve the above system
Since we can find infinite such vectors this system has infinitely many solutions
This is because the rows of the corresponding matrix are linearly dependent
A square matrix \(A\) is nonsingular if and only if the rows and columns of \(A\) are linearly independent
To every square matrix we can assign a unique number called the determinant.
For \(2 \times 2\) matrices, the determinant is given by,
\[ \begin{bmatrix} \color{red}{a} & \color{blue}{b} \\ \color{blue}{c} & \color{red}{d} \end{bmatrix} = {\color{red}{ad}} - {\color{blue}{bc}} \]
\[ det \begin{bmatrix} a & b & c \\ d & e & f \\ g & h & i \end{bmatrix} = a(ei - fh) - b(di - fg) + c(dh - eg) \]
a square matrix \(A\) has a nonzero determinant, if and only if it possesses an inverse matrix \(A^{-1}\), with the property that \(A A^{-1} =A^{-1} A = I\).
As a consequence, if we pre-multiply both sides of \(Ax = b\) by \(A^{-1}\), we get
\[ x = A^{-1} b \]
This is the solution to \(Ax = b\) — the solution we are looking for
\(A_{n \times k}\) matrix with \(n > k\)
this is important in many settings, including linear regression
Given arbitrary \(y \in \mathbb R ^n\), we seek an \(x \in \mathbb R ^k\) such that \(y = Ax\)
focusing on the case where the columns of \(A\) are linearly independent.
\(A_{n \times k}\) matrix with \(n < k\)
there are either no solutions or infinitely many — in other words, uniqueness never holds
For example, consider the case where \(k=3\) and \(n=2\)
\[ y = x_1 (\alpha a_2 + \beta a_3) + x_2 a_2 + x_3 a_3 = (x_1 \alpha + x_2) a_2 + (x_1 \beta + x_3) a_3 \]
Let \(A\) be a symmetric \(n \times n\) matrix
\(A\) is
Analogous definitions exist for negative definite and negative semi-definite matrices
if \(A\) is positive definite, then all of its eigenvalues are strictly positive